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While  we  may  have  the  Intuitive  idea  of  one  programming  language  having  greater 
power  than  another,  or  of  some  subset  of  a language  being  an  adequate  "core" 
for  that  language,  we  find  when  we  try  to  formalize  this  notion  that  there  is 
a serious  theoretical  difficulty.  This  lies  in  the  fact  that  even  quite  rudi- 
mentary languages  are  nevertheless  "universal"  In  the  following  sense.  If  the 
language  allows  us  to  program  with  simple  arithmetic  or  list-processing  functions, 
then  any  effective  control  structure  can  be  simulated,  traditionally  by  encoding 
a Turing  machine  computation  In  some  way.  In  particular,  a simple  language  with 
some  basic  arithmetic  can  express  programs  for  any  partial  recursive  function. 

Such  an  encoding  is  usually  quite  unnatural  and  Impossibly  Inefficient.  Thus  in 
order  to  carry  on  a practical  study  of  the  comparative  power  of  different  languages 
we  are  led  to  banish  explicit  functions  and  deal  instead  with  abstract,  unin- 
terpreted programs,  or  schemas.  What  follows  is  a brief  report  on  some  preliminary 
exploration  in  this  area. 

This  report  describes  research  done  at  the  Artificial  Intelligence  Laboratory  of 
the  Massachusetts  Institute  of  Technology.  Support  for  the  laboratory's  artificial 
intelligence  research  Is  provided  In  part  by  the  Advanced  Research  Projects  Agency 
of  the  Department  of  Defense  under  Office  of  Naval  Research  contracts  N00014-75-C- 
0643  and  N00014-75-C-0522. 
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Introduction 

While  we  may  have  the  Intuitive  Idea  of  one  programming  language  having  greater 
power  than  another,  or  of  some  subset  of  a language  being  an  adequate  'core'  for  that 
language,  we  find  when  we  try  to  formalize  this  notion  that  there  Is  a serious  theoret- 
ical difficulty.  This  lies  In  the  fact  that  even  quite  rudimentary  languages  are  never- 
theless 'universal'  in  the  following  sense.  If  the  language  allows  us  to  program  with 
simple  arithmetic  or  list-processing  functions  then  any  effective  control  structure  can 
be  simulated,  traditionally  by  encoding  a Turing  machine  computation  In  some  way.  In 
particular,  a simple  language  with  some  basic  arithmetic  can  express  programs  for  any 
pertlal  recursive  function.  Such  an  encoding  la  usually  quite  unnatural  and  Impossibly 
Inefficient.  Thus  In  order  to  carry  on  a practical  study  of  the  comparative  power  of 
different  languages  we  are  led  to  banish  explicit  functions  and  deal  Instead  with  ab- 
stract, uninterpreted  programs,  or  schemas . What  follows  Is  a brief  report  on  some  pre- 
liminary exploration  In  this  area. 

Languages 

The  simplest  language  we  ehall  study  Is  a flow-chart  language  with  which  we  write 
program  schemas  such  as  shown  In  Figure  1.  When  we  have  provided  an  Interpretation  for 


Figure  1.  A rrogtam  lehaam. 
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1 lie  basic  I unction  .symbols  and  predicate  symbols  Involved,  a program  schema  may  be  re- 
g.irded  111  an  executable  program  defining  a partial  function.  As  a convenient  convention 
we  lake  the  arguments  of  this  function  to  be  the  Initial  values  of  locations  Lj , 1,^,  ... 

(up  to  as  many  as  are  mentioned)  and  the  value  as  being  the  final  value  of  1^  If  this  is 
defined.  Other  locations  Mq,  M^,  •••,  Nq,  N^,  are  used  only  as  'working  space'.  We 
may  as  well  assume  that  none  of  these  latter  locations  Is  used  as  an  argument  before 
being  assigned  to. 

Another  language  we  shall  use  provides  for  the  recursive  definition  of  functions 
using  conditional  expressions.  For  example: 

f(Xj,  Xj)  * If  g(x2)  then  F(lf  P(x^)  then  X2  else  A) 
else  if  Pfxj)  then  true  else  f(x2,  F(x^)) 

g (x | ) » If  P(f(xlt  F(x1))  then  false  else  g(A) 

defines  two  abstract  functions  by  simultaneous  recursion.  Given  an  Interpretation  of  the 
basic  function  and  predicate  symbols  (which  are  the  upper-case  symbols  In  the  definitions), 
such  a system  defines  a partial  function  corresponding  to  each  equation.  The  system  is 
deemed  to  compute  the  function  given  by  the  first  definition.  Such  a system  Is  celled 
a recursive  schema.  If  there  la  only  one  equation  It  Is  a simple  recursive  schema. 

If  several  It  Is  compound . 

Interpretations  and  Equivalence 

An  Interpretation.  I,  for  program  schemas  and  racuralva  schemas  provldae: 

(I)  a domain  D 

(II)  for  each  basic  predicate  symbol  P,  a (total)  predicate 

Pj  : D -*  {true,  false} 

(ill)  for  each  n-ary  basic  function  symbol  (n  * 0)  F,  a (total  function) 

Ft  : d"  -*  D 

In  general  we  use  P,  P',  •••  as  predicate  symbols  and  othar  capital  letters  as  function 
symbols.  A constant  Is  a 0-ary  function. 

The  partial  function  defined  by  schema  S under  Interpretation  I Is  denoted  by  Sj. 

Fur  two  partial  functions  u,  v,  we  write  u * v If,  for  all  x,  either  both  of  u(x),  v(x) 

.ire  undefined  or  both  are  defined  and  u(x)  - v(x).  TWO  schemas  S,  S'  (not  necessarily 
of  the  same  type)  are  '.rongly)  equivalent,  S 3 S'  If,  for  all  Interpretations  I, 

s.  ~ si" 

l-et  Jf  , jA ' be  two  classes  of  schemas.  We  write  J < j ' If 
(VS  (■  J ) (MS ' f ^')(S  * s')  and  A < J ’ If  Jf  < J'  but  not  J'  < Jl  . Let  be  the 
class  of  recursive  schemas  and  p be  the  class  of  program  schesws. 
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Theorem  1.  f>  < & . 

Proof.  To  show  P < fi.  involve*  only  a routine  conatruction  which  we  outline  beluw. 

(For  further  details  see  [1].)  For  each  flowchart  box  bt  in  program  schema  U,  consider 
the  abstract  partial  function  fj,  which  for  given  values  of  all  the  locations  is  com- 
puted aa  follows.  Start  the  schema  at  b^  with  the  locations  having  these  values,  and 
then  the  valua  of  the  function  is  the  final  value  of  Lq,  if  defined.  It  is  easy  to 
write  a compound  recursive  schema  which  defines  the  f^  recursively.  If  b^  is  the 
START  box  then  the  function  corresponding  to  U is  got  from  fQ  by  simply  suppressing 
some  of  its  arguments. 

Now  to  show  that  the  containment  is  proper,  consider  the  followli^g  recursive 
schema  V. 

V.  f(x)  ■ If  P(x)  then  x 

else  H(f(L(x)),  f(R(x))  ) 

We  shall  show  that  no  program  schema  can  be  equivalent  to  V. 

A useful  notion  Is  that  of  a free  Interpretation.  In  a frea  interpretation  the 
domain  is  the  set  of  all  strings  composed  from  the  basic  function  symbols.  Then,  for 
example,  if  H is  a binary  function  symbol  and  E^,  E2  are  strlnga, 

Hj(E  , Ej)  - H E^  E^  for  any  free  Interpretation  I. 

The  Interpretation  of  the  predicate  symbols  Is  unconstrained.  It  should  be  clear  that 
for  most  purposes,  It  Is  sufficient  to  consider  only  free  interpretations.  For  example, 
for  any  S,  S',  S ■ S'  if  and  only  If  Sj  - Sj  for  all  free  I. 

For  the  present  proof  (and  the  naxt  one)  we  consider  the  family  of  frae  Interpre- 
tations C I0 ) n > 0 where 

PT  (E)  ■ true  If  length(E)  ■ n 
ln 

- false  otherwise. 

What  Is  the  value  of  V.  (X)  where  X Is  the  empty  string? 
ln 

For  n - 1,  the  value  Is  HLR, 
for  n - 2,  It  Is  H HLLR  LHLRRR 
etc. 

We  show  that  to  compute  the  value  for  In  at  least  n + l locations  are  required  for 
working  space.  It  helps  to  present  the  same  situation  In  a geometric  form.  We  ere  to 
play  a game  of  placing  movable  tokens  on  a finite  binary  tree,  TB.  For  n * 3,  the  tree 
T3  Is  given  ln  Figure  2.  The  rules  of  the  gams  are  that  any  token  may  be  put  on  a bot- 
tom node  at  any  time.  If  the  two  nodes  bslow  a given  node  era  covered  then  any  token 
nay  be  put  at  the  given  node.  Haw  many  tokens  are  needed  to  be  able  to  reach  the  top 
node?  It  Is  easy  to  saa  that  n ♦ 1 ere  sufficient  In  general  for  Tn,  and  the  following 
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Ftgur*  2.  Binary  tret  for  n • 3. 

argument  establishes  their  neceaalty. 

A tree  will  be  said  to  be  closed  at  a given  stage  of  the  game  If  there  Is  at  least 
one  token  on  each  path  from  top  to  bottom.  Initially,  the  tree  la  devoid  of  tokens  and 
so  is  not  closed.  Finally  there  Is  a token  at  the  top  node  and  so  It  la  closed.  We 
concentrate  our  attention  on  the  time  at  which  the  tree  first  becomes  closed.  This  can 
only  happen  as  a result  of  placing  a token  at  a bottom  node,  closing  off  the  last  path. 
Now  this  path  Is  otherwise  empty  of  tokens  and  so  aach  of  the  n sub-trees  sprouting  Im- 
mediately off  this  path  must  be  Independently  closed.  Since  at  least  one  token  Is  needed 
to  close  any  tree,  there  are  at  least  n+1  tokens  on  the  tree  at  this  time. 

The  relation  between  a computation  of  a program  schema  and  moves  In  this  game  we  as- 
sume to  be  self-evident.  Suppose  a program  scheme  U Is  equivalent  to  V and  has  just  r 

locations.  This  Is  Immediately  absurd  because  there  Is  no  way  for  U to  compute  the  re- 

quired output  for  the  Interpretation  Ir.  This  completes  the  first  proof. 

Program  schemas  are  therefore  unequal  to  the  task  of  computing  certain  abstract 
functions  for  the  simple  reason  that  a fixed  number  of  locations  cannot  compute  all  the 
necessary  final  values.  This  Is  not  a very  Interesting  reason  so  we  show  In  a second 
proof  of  the  same  theorem  that  program  schemas  can  fall  for  more  subtle  reasons  which 

reflect  the  Inadequacy  of  their  control  structure.  The  "target"  schema  for  this  proof 

computes  a partial  predicate  and  Is  got  from  V,  more  or  less,  by  replacing  H by  the 
Koolean  function  and. 

Second  Proof  of  Theorem  .1.  Consider  the  recursive  schema: 

W.  f(x)  - If  P(x)  then  true 

else  If  f (L(x))  then  f(R(x))  else  false 

Since  the  value  of  this  schema  Is  either  true  or  Is  undefined,  no  argument  of  the 
kind  given  above  Is  useful.  Suppose  program  schema  U Is  equivalent  to  W and  has  t boxes 
and  r locations,  and  we  may  suppose,  without  loss  of  generality,  that  the  only  predicate 
ami  function  symbols  occurring  In  U are  P,  L,  R.  A state  of  U under  a given  Interpre- 
tation Is  specified  by  a box  of  U together  with  a value  from  the  domain  for  aach  loca- 
tion. TVo  states,  Sp  S2  are  (Just-for-now)  equivalent  If  the  sequences  of  boxes  for  the 
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computations  continuing  from  S^,  S2  are  the  same.  Consider  the  equivalence  classes  of 
states  of  U under  the  interpretation  I defined  above.  With  a little  thought  one  can 
see  that  the  only  property  of  the  value  of  a location  which  can  affect  the  equivalence 

class  is  the  length  of  the  string.  Furthermore  any  two  strings  of  length  greater  than  n 

are  indistinguishable  by  P either  immediately  or  in  their  future.  ITius  1)  has  at  most 
t*(n  + 2)r  equivalence  classes  under  I If,  during  a computation  of  U an  equivalence 
class  is  repeated,  then  the  computation  is  doomed  to  loop,  supposed  to  ter- 

minate (with  value  .true),  hence  must  never  repeat  an  equivalence  class,  and  therefore 
runs  for  no  more  than  t *(n+2)r  steps.  If  n has  been  chosen  sufficiently  large  that 

2n  > c • (n  + 2 )f 

Chen  U cannot  have  time  to  test  each  one  of 

P(L  L •••  L),  P(R  L • • • L),  • • • • , P(R  R • • • R) 

but  neverthelesa  halts  with  value  true.  If  we  slightly  modify  In  by  letting  P be  false 
for  some  such  expression  which  Is  not  tested,  then  of  course  U will  never  notice  and  still 

give  the  value  true . but  the  value  of  W Is  now  undefined.  Thus  U f W and  the  proof  is 

complete.  (Of  course  this  method  of  proof  works  equally  well  for  schema  V.) 

Linearly  Recursive  Schemas 

The  task  of  effectively  characterising  those  recursive  schemas  which  are  "program- 
mable” Is  Impossible.  Indeed,  if  P*  £ A Is  the  subset  of  those  recursive  schemas 
which  are  equivalent  to  some  program  schema,  we  can  neither  effectively  enumerate  f*  nor 
ft.  - This  follows  from  the  application  of  straightforward  techniques  which  are 

described  In  [1]  or  [2]  and  will  not  be  proved  here.  The  best  that  we  can  hope  for  then 
Is  to  effectively  characterise  large  sub-classes  of  ft  and  ft.  - ft  . At  present  we 
have  a fairly  extensive  effective  sub-class  of  ft.  - ft  which  encompasses  both  V and  W 
and  which  we  are  seeking  to  extend  further. 

On  the  other  hand,  an  approximation  to  ft  * is  provided  by  the  "linear"  schemas 
which  are  described  shortly.  A recursive  schema  may  be  regarded  as  giving  a way  of 
computing  the  value  of  a non-baslc  function  given  the  values  of  the  same  or  other  func- 
tions at  other  arguments.  The  determination  of  these  latter  values  will  in  general  re- 
quire further  recursive  calls,  and  so  on.  For  example,  in  the  following  schema: 

f(x,  y)  * if  P(x)  then  f(x,  S(y))  else 

If  P(g (R(x)))  then  S(x)  else  ll(g(R(x)),  R(y)) 

g(x)  - If  P'(x)  then  K(g (S(x)),  R(g(S(x)))) 
else  if  P(x)  then  f(x,  x)  else  A 
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die  evaluation  of  f(x,  y)  might  need  the  value  of  f(x,  S(y))  or  the  value  of  g(R(x))( 
and  the  evaluation  of  g(x)  might  need  Che  value  of  g(S(x))  or  f(x,  x).  But  notice  In 
thl9  example,  and  thla  la  the  defining  criterion  for  a linear  achama,  that  In  any  eval- 
uation at  moat  one  further  value  of  a non-baatc  function  la  Immediately  required. 

Theorem  2.  If  X la  linear  then  X € P . 

Rather  than  give  a proof  of  thla  theorem,  we  ahall  give  here  an  example  of  the 
translation  Into  program  achema  form  of  a aimple  linear  program.  The  proof  of  Theorem  2 
however,  la  by  reducing  the  general  linear  caae  to  a achema  laomorphlc  to  thla  example. 

So  the  following  translation  or  compilation  may  be  regarded  aa  the  "canonical"  example! 

L?  f(x)  = If  P(x)  then  R(x)  elae  S(f(T(x)),  x) 

The  second  argument  of  S la  an  Important  feature,  adding  to  the  difficulty  of  thla  ex- 
ample. Let  v. , v.,  •••  v be  the  valuea  of  the  aucceaalve  lnatancea  of  f called  for  In 
0 1m 

some  terminating  evaluation.  We  obaerve  that: 
v ' R (TjB*(x)) 

vr  - Sjfv^,  T<r)(x))  for  r - 0,  m-l, 

and  further, that  If  only  we  had  eome  form  of  counter  able  to  count  up  to  lntegera  leaa 

than  or  equaL  to  m,  then  we  could  eaally  evaluate  f by  computing  In  turn 

v , v vA.  However,  auch  a counter  can  be  simulated  when  we  realise  that: 

in  ni"  I 0 

P fT[r)(x))  - falae  for  r - 0,  •••,•- l 

- true  for  r ■ m. 

An  equivalent  program  acheaia  to  L la  given  In  Figure  3. 

Discussion 

Of  course  the  efficiency  of  the  above  translation  leavea  something  to  be  desired, 
and  we  are  paying  for  our  restriction  to  only  a finite  nuafcer  of  locations  by  an  Increase 
In  the  computation  time.  The  cause  of  the  problem  In  thla  example  is  that  we  are  re- 
quired to  compute.  In  order,  the  sequence: 

T(m)(x),  T(m‘l)(x),  TT(x),  T(x) 

which  the  schema  L does  by  computing  each  term  Independently  all  the  way  up  from  x,  and 

A 

thus  requires  of  the  order  of  m operations.  To  Investigate  this  situation  of  trading 
off  between  time  and  spaca  In  more  detail  we  heve  considered  the  following  simple  com- 
binatorial problem. 
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Program  ichema  equivalent 
to  e linear  echeaa. 


Suppose  we  have  always  available  the  value  X In  a "read-only"  location,  MQ,  and  we 
have  In  addition  a further  k locations,  Mj,  ’•*,  M^.  For  given  fixed  m,  we  went  to 
write  • shortest  possible  sequence  of  Instructions  of  the  form: 

Ht  : - T(Mj ) 0<isk,0<]«k 

to  compute  successively  the  values 

T(m)  (X),  •••  T T(X),  T(X). 

The  values  may  appear  In  any  location,  but  must  be  produced  In  the  given  order.  Let 
L(k,  m)  be  the  length  of  the  shortest  such  sequence  of  Instructions.  For  example, 

L(l,  m)  - jm  (mfl).  We  have  derived  an  explicit  formula  for  L(k,  m)  from  which  we 
can  show 

t n.  _l+l/k 

L(k,  n)/vn 

Thus  there  exists  the  possibility  of  computing  linear  recursive  schematic  functions 
using  a fixed  number  of  locations  with  reasonable  efficiency.  However,  It  Is  likely 
that  some  more  versatile  control  structure  than  that  of  program  schemas  Is  required  to 
realise  this  possibility. 
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We  are  currently  Investigating  the  relations  between  various  augmented  forms  of 
program  schema,  such  as  program  schemas  with  counters,  push-down  stores,  or  stacks. 

These  augmentations  may  either  be  expressed  by  more  complicated  definitions  of  'schema' 
■•or  else  may  be  got  from  the  ordinary  program  schema  by  fixing  part  of  the  Interpretation. 

I For  example,  to  express  the  Idea  of  counters  we  could  fix  the  Interpretation  of  the  con- 
[ sjtant  symbol  ZERO,  function  symbols  ADDl,  SUB1,  and  predicate  symbol  POSITIVE?  In  the 
(J>vloiis  way.  We  must  take  care  In  applying  some  of  the  well-known  results  of  automata 
cyheory  to  this  area.  For  example,  program  schemas  with  counters  are  not  'universal'  In 
^his  theory,  because  the  first  proof  of  Theorem  1 shows  that  no  such  schema  can  be  equiva- 
lent to  the  recursive  schema  V.  However,  we  can  show  that  under  an  appropriate  deflnl- 
:ion  of  push-down  store  program  schemas,  they  are  equivalent  to  the  class  of  recursive 
^schemas . 


as  we  allow  a parallel  form  of  con- 


s.A  Class  of  Parallel  Schemas 

For  a very  simple  extension  of  recursive  sche 
idltional  expression. 

IF  p THEN  g ELSE  r 

has  the  value  g If  p Is  true,  the  value  r If  p Is  false  and  In  addition  if  p Is  undefined 
but  g and  r are  both  defined  and  are  equal  then  the  value  is  g.  As  a special  case  of  this 
connective  we  define: 

p /OR/  g to  Man  IF  p THEN  true  ELSE  g 

So  If  either  p or  g has  the  value  true  then  (p/OR/g)  Is  true,  and  If  both  p and  g are 
undefined  It  Is  also  undefined.  Let  J be  the  class  of  schemas  we  get  by  extending  ft. 

with  /OR/. 


Theorem 


3.  (i<J  • 


The  proof  of  this  theorem  which  Is  too  long  and  detailed  to  give  here  Is  by  showing 
that  no  recursive  schema  can  be  equivalent  to  the  following  schema. 


S. 


f(x)  - if  P(x)  then  true  else  (f (L(x))/OR/f (R(x))) 


Tin-  value  of  Sr(M  for  a free  Interpretation  1 Is  true  If  there  is  at  least  one  string 

of  L's  and  R's  for  which  P Is  true  and  Is  undefined  otherwise.  In  the  proof  we  char- 

* 

actor tze  the  behavior  of  recursive  schemas  under  the  free  Interpretation  where  P Is  al- 
ways false.  The  only  strings  on  the  binary  tree  of  (L,  R)-strlngs  that  a recursive 
schema  can  "look  at"  are  those  strings  within  e bounded  dlstence  of  a finite  number  of 
paths  descending  through  the  tree.  Therefore,  any  recursive  schema  must  fall  to  test 
certain  strings  and  so  will  give  e different  velue  from  S for  some  lnterpretetlons . 
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Cone  Ins  ion 

We  have  a clear  notion  of  effective  schemes  of  computation  Involving  uninterpreted 
functions,  and  should  like  to  have  a fairly  natural  augmentation  of  program  schemas 
capable  of  representing  any  such  effective  computation.  A good  candidate  for  tills  su- 
preme position  In  the  helrarchy  of  schemas  would  seem  to  be  program  schemas  wl  tli  two 
push-down  stores.  Provided  the  schema  has  the  ability  to  put  special  control  constants 
In  Its  stores  and  to  subsequently  recognize  them,  the  "universality"  of  this  model  ap- 
pears assured. 

Throughout  this  paper  we  have  made  the  simplifying  assumption  that  only  total  func- 
tions and  predicates  can  appear  in  interpretations.  Removing  this  assumption  changes 
several  of  our  results,  and  Introduces  new  considerations.  For  example,  to  show  that 

J we  have  only  to  notice  that  the  schema 

f(x)  = (P(x)  /OR/  P'(x)) 

has  no  recursive  equivalent.  Also  the  concept  of  effective  computation  Is  now  no  longer 
unambiguous  and  depends  on  the  conventions  we  adopt  concerning  the  evaluation  of  partial 
basic  functions. 
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